A Methodology for Extrinsically Evaluating Information Extraction Performance
نویسندگان
چکیده
This paper reports a preliminary study addressing two challenges in measuring the effectiveness of information extraction (IE) technology: • Developing a methodology for extrinsic evaluation of IE; and, • Estimating the impact of improving IE technology on the ability to perform an application task. The methodology described can be employed for further controlled experiments regarding information extraction.
منابع مشابه
Developing and evaluating an IT specification extraction system
Purpose – This paper seeks to establish an extraction system for an information technology (IT) product specification named ITSIES which combines the natural language process (NLP) with the ontology concept and also to evaluate the system’s effectiveness in advance. Design/methodology/approach – The development of the system is based on a prototype design and performance validation. This study ...
متن کاملEvaluating a Natural Language Dialogue System: Results and Experiences
Dialogue Management components are becoming increasingly useful in Natural Language Information Retrieval systems, to aid users find information they require. These components need to be evaluated, both intrinsically, to gauge development approaches, and extrinsically to measure subjective performance. We introduce a Natural Language Dialogue System, the YPA, developed at the University of Esse...
متن کاملA New Method for Improving Computational Cost of Open Information Extraction Systems Using Log-Linear Model
Information extraction (IE) is a process of automatically providing a structured representation from an unstructured or semi-structured text. It is a long-standing challenge in natural language processing (NLP) which has been intensified by the increased volume of information and heterogeneity, and non-structured form of it. One of the core information extraction tasks is relation extraction wh...
متن کاملEvaluating Dependency Representations for Event Extraction
The detailed analyses of sentence structure provided by parsers have been applied to address several information extraction tasks. In a recent bio-molecular event extraction task, state-of-the-art performance was achieved by systems building specifically on dependency representations of parser output. While intrinsic evaluations have shown significant advances in both general and domain-specifi...
متن کاملA review on EEG based brain computer interface systems feature extraction methods
The brain – computer interface (BCI) provides a communicational channel between human and machine. Most of these systems are based on brain activities. Brain Computer-Interfacing is a methodology that provides a way for communication with the outside environment using the brain thoughts. The success of this methodology depends on the selection of methods to process the brain signals in each pha...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2005